Using KMF for HMAC verification

#Security , #KMF , #HMAC

Cheng Chen Liu

11 minute read

Using KMF for HMAC Verification

Hash-based message authentication code (or HMAC) is a common way to validate the authenticity of messages sent over the internet. For example, GitHub and Gitea rely on HMAC (SHA256) to secure webhooks. My team wanted to sync our Gitea Issues and Pull Requests to our ServiceNow instance, thus we wanted a Scripted REST API that verifies HMAC for those webhook messages.

There have been a few posts describing how HMAC verification can be implemented (e.g., using CryptoJS and GlideCertificateEncryptionAPI) on ServiceNow instances; since the Key Management Framework (or KMF) has a method to validate HMAC, we decided to explore this newer alternative.

There are several benefits of using KMF for cryptographic operations, such as segregation of roles, purposes, and lifecycles, as listed in the official documentation here. In our case, we are using it for safely storing the HMAC secret key since sys_property is not ideal for this purpose.

Since this was our first attempt using KMF, we also needed to complete some preparative operations before getting to the HMAC part. The first half of this post covers many generic KMF setups, such as setting up roles, and importing keys; details specifically about using KMF for HMAC start in the second half of this post.

What is KMF?

According to the official document, KMF “provides a secure and comprehensive interface for instance-side cryptographic key management services.” It offers so many capabilities that it looks complicated at first. We gathered the following rough summary:

  • KMF is managed by individual Modules, usually one module per scoped app or usage, etc.

  • A Module can contain several Specifications for different purposes (e.g., symmetric authentication, encryption, signing, etc.)

  • Keys are stored under Modules and linked to the Specifications

    • Each key has its own lifecycle definition (such as “Revoked”, “Activated”, or “Generated”)

    • Keys can be generated by ServiceNow or uploaded by users

  • Modules are governed by Access Policies, which allow users to define specific usages

Pre-requisite: Plugins and Roles

Before starting, we had to ensure the instance had KMF enabled (this should be the default from San Diego onwards). In our case, we just updated our instance from Rome; additional support requests (e.g., “Activate Plugin” on Support) were required to ensure that KMF was enabled properly.

To make sure the plugin was installed, we first checked on the Plugins page (System Definition > Plugins) and ensured the following were installed and activated:

  • Encryption Core (com.glide.encryption.core)

  • Key Management Framework (com.glide.kmf.global) Then, we checked the KMF health-check page (Key Management > Diagnostics). Initially, our instance showed the Key Secure and Hardware Security Module (HSM) features were both in the “MALFUNCTION” state, so we raised a request via Support to get them enabled correctly. Once all the Key Management Framework Health checks were in the “OPERATIONAL” state, we were ready to accommodate our HMAC use case.

health-check.png

The next pre-requisite was to add roles required for using KMF following this guide: Key Management Framework roles. This was a two-part process:

Part 1: We had to make a KMF admin user. We were using our instancebs security_admin user (with admin and security_admin roles). Then we had to:

  1. Elevate to security_admin

  2. Navigate to System Security > Key Management Administration, select the user to be assigned with Key Management Admin role. These changes add the sn_kmf.admin role to the user. (Note, assigning this role was done differently than directly editing a user’s role.) Note that the sn_kmf.admin role itself is not for working on KMF: it does not grant the privilege to perform actual KMF operations, like creating modules or changing keys. It is just a role to assign other sn_kmf.* roles (see Part 2 below).

Part 2: We then assigned other roles required for KMF operations:

  1. As the sn_kmf.admin user, go to the user’s form and edit roles

  2. Add roles: sn_kmf.cryptographic_operator and sn_kmf.cryptographic_manager.

  • Note that other regular admin users without sn_kmf.admin role will not see the sn_kmf.* roles in the role list.

kmf_role.png

Now, we had enabled the necessary KMF features and had a user with the required roles (sn_kmf.cryptographic_manager and sn_kmf.cryptographic_integrator), we could start creating our KMF module for HMAC.

Creating the KMF Module and Uploading a Custom Key

Here begins the actual KMF operations for our HMAC usecase. As an overview, the activities required to accomplish this include:

  1. Creating the Cryptographic Module and Specification

  2. Setting up the instance-level Import Key (if needed)

  3. Uploading the HMAC key

  4. Configuring Access Policies

Creating the Cryptographic Module and Specification

First, we created a Module and a Specification to store our HMAC key with this procedure:

  1. Navigate to Key Management > Cryptographic Modules > All, create a new Cryptographic Module. Select the appropriate Application (scope) and provide a Module name.

  2. When inside the crypto module, navigate down to Crypto Specifications and create “New.”

image-1649803915342.51.51-pm.png

  1. Inside the new Crypto Specification, set the Crypto purpose to “Symmetric Authenticity”, and choose the algorithm (e.g., “HMAC 256 SHA 256”), click “Next”.

image-1649803957439.52.32-pm.png

  1. On the second page, Lifecycle Definition, leave the default settings in place, and click “Next.”

  2. On the third page, Key Origin, Origin is “Import from web service”, Key alias is the HMAC key’s alias (it may not really be checked later).

    • Unfortunately, uploading the key is not done simply on this page, see the following two sections for key import procedures. For now, this just creates a placeholder.

  3. On the last page, verify all settings have been entered correctly. Note down the sys_id of this Crypto Spec (right click on the form header and “Copy sys_id”).

Setting up the Instance Import Key

As a first-time KMF user, before uploading the actual HMAC key (which is covered in the next step below), we needed to set up the instance-level Import Key. If the instance already has an import key, this step can be skipped.

As described in the official document, when uploading any customer keys to the instance, the keys are required to be wrapped by this “Import Key” first. Setting up the import wrapping key should be a one-time operation; following this, importing other KMF keys will use this same instance-level Import Key for wrapping.

We used the following procedure for generating the instance-level Import Key pair on a local workstation:

  1. Generate a key pair with RSA 4096 Algorithm:

a) openssl req -x509 -newkey rsa:4096 -keyout privatekey.pem -out certificate.pem -days <expiration_days>

b) Choose a reasonable expiration_days; if the key expired, you may need to re-do this whole section of setting up the instance-level import key.

c) Note down the passphrase before moving on.

  1. Convert it to pkcs12 format:

a) openssl pkcs12 -export -inkey privatekey.pem -in certificate.pem -name testimportkey -out cert.pfx

b) Note down the alias (followed by -name, e.g., “testimportkey”) and the export password.

  1. Store the key files securely.

We then followed this procedure for uploading the wrapping key:

  1. Navigate to Key Management > Import Settings > Key Import Settings.

  2. In the Algorithm Definition section, verify the Crypto Purpose is set to Asymmetric Key Unwrapping.

  3. For Algorithm, select RSA 4096 to align with our asymmetric key material for the imported keystore.

wrapping_key.png

  1. Click “Next.”

  2. In the Lifecycle Definition section, leave as default and click “Next” to continue.

  3. In the Key Origin section

a) Select “Import from PKCS12” as the Origin

b) For Key Alias, enter the alias associated with the generated import key (e.g., “testimportkey” in our example)

  1. Click “Next.”

  2. In the Key Creation Section, click on “Import Key” and select the keystore file (i.e., cert.pfx from above) to upload

a). Enter the keystore filebs export password here

import_key_upload.png

Completion of this section should result in the successful creation and configuration of an instance-level Import Key.

Uploading the HMAC Key

On to the second part of “Import a key from web service”, we used this procedure to create and upload our HMAC key:

  1. Create a text key for HMAC (because Gitea wants a secret in plain text)

a) echo -n 'whateveryourkeyis...' > key.txt

b) Note: the -n option removes the newline character at the end

  1. Wrap the key file with the instance’s Import Key

a) Extract the public key from the Import Key’s certificate.pem: openssl x509 -pubkey -noout -in certificate.pem > pubkey.pem

b) Wrap the key: openssl pkeyutl -encrypt -pubin -inkey pubkey.pem -in key.txt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 -out key.wrapped

c) If the above command fails with complaints about pkeyopt, it is likely because of some an issue with local workstationbs openssl. Try to do it in another Linux environment (e.g., pull a Ubuntu docker image and perform the commands there)

  1. Upload the key via a POST request. We used Postman to do it:

a) Select POST as the method

b) Put in the URL for KMF key import, e.g., https://<myinstancename>.service-now.com/api/sn_kmf/key/import

c) In the Params tab, add a key cryptoSpecSysID, and the value needs to be the HMAC Crypto Specificationbs sys_id. This should update the URL to https://<myinstancename>.service-now.com/api/sn_kmf/key/import?cryptoSpecSysID=<crypto_spec_sys_id>

d) In the Authorization tab, select Type as Basic Auth, and input the username/password. We used a local account to do this, and needed to simply append the OTP code right after the password.

e) In the Body tab, select binary, choose the wrapped key file (i.e., key.wrapped) as binary data in body

f) Send the POST request. When successful, the instance responds with a 200 OK

postman.png

After successfully completing this section, the instance was configured with a Cryptographic Spec for HMAC that holds our customized key, and it’s almost ready to be used. Just one more thing.

Configuring Access Policies

Initially, we did not configure the access policy correctly, so the default behavior rejected our usage and threw error messages like “Access Denied to cryptographic module bx_snc_appname.modulenameb”.

To fix this, users should navigate to “Module Access Policies” (sys_kmf_crypto_caller_policy), then filter for policies applied to the crypto module. There might be none, thus all access were denied. Create or modify the records here accordingly.

access.png

The access control is quite specific. We created a policy for background scripts (for testing purposes), and another policy for the Script Include used by the Scripted API.

Using KMF for HMAC Verification

Finally, we were ready to use our key in background scripts to do HMAC verification! The KMF API documentation is quite helpful, it can be found here: KMFCryptoOperation API - Scoped, Global

Testing Script

The following is a simple background script (i.e. run from /sys.scripts.do) for testing purposes. Both inputs to KMF operations need to be supplied as base64-encoded strings.

var text = 'hello';
var b = GlideStringUtil.base64Encode(text);
var expected64 = 'lVKY8yRGtYurZ6D1lpLovyndUxQY0uT8rWMAM2L56zw='; // replace this

gs.info("Text: " + text + ", base64: " + String(b));

var op = new sn_kmf_ns.KMFCryptoOperation('x_snc_appname.modulename', 'MAC_VERIFICATION');
op.withAdditionalInput(expected64);
var res = op.doOperation(String(b));

gs.info(res); // true or false

When succeed, the above script should generate the following output:

*** Script: Text: hello, base64: aGVsbG8=
*** Script: true

The above script performs the MAC_VERIFICATION operation; alternatively, KMF can perform MAC_GENERATION. But note the following:

  • Add op.withOutputFormat('KMFBASE64'), or the output will be in “FORMATTED” format and includes some encrypted meta info at the beginning

  • The KMFBASE64 format uses the web-safe base64 symbols (e.g., it uses ‘-’ and ‘_’ instead of ‘+’ and ‘=’).

Other Helpful Commands

In addition, the following openssl commands might be helpful for testing and debugging:

  • Unwrapping a key: openssl pkeyutl -decrypt -inkey privatekey.pem -in mykey.wrapped -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256

  • Generate a HMAC with plaintext key: echo -n "hello" | openssl dgst -sha256 -hmac whateveryourkeyis -binary | openssl enc -base64 -A

Script Include for HMAC Verification

The following is an example Script Include that handles HMAC verification for our webhook API. Since Gitea sends hex strings, additional helper functions (e.g., convertToBase64()) are required to convert to base64 for KMF operations.

var GitUtil = Class.create();
GitUtil.prototype = {
    initialize: function(payload) {
    },

    //////////////////////////////////////////////////////////
    // Helpers to validate payload HMAC
    //////////////////////////////////////////////////////////
    validateSignatureKMF: function(payload, headerSignature) {
		var payloadB64 = gs.base64Encode(payload);
	     var expected = this.convertToBase64(headerSignature); // Gitea sent hex
		gs.debug("Signature is: " + expected);

		var op = new sn_kmf_ns.KMFCryptoOperation('x_snc_appname.modulename', 'MAC_VERIFICATION');
		op.withAdditionalInput(expected);
		var result = op.doOperation(String(payloadB64));
		if (!result){
			var op2 = new sn_kmf_ns.KMFCryptoOperation('x_snc_appname.modulename', 'MAC_GENERATION');
			op2.withOutputFormat("KMFBASE64");
			var actual = op2.doOperation(String(payloadB64));
			gs.warn("Invalid signature: " + expected + " vs " + actual);
		}
		return result;
    },

    convertToBase64: function(myHexString) {
        var hexArray = myHexString.toString()
            .replace(/\r|\n/g, "")
            .replace(/([\da-fA-F]{2}) ?/g, "0x$1 ")
            .replace(/ +$/, "")
            .split(" ");
        var byteString = String.fromCharCode.apply(null, hexArray);
        var base64string = this.btoa(byteString);
        return base64string;
    },

    btoa: function(bin) {
        var tableStr = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
        var table = tableStr.split("");
        for (var i = 0, j = 0, len = bin.length / 3, base64 = []; i < len; ++i) {
            var a = bin.charCodeAt(j++),
                b = bin.charCodeAt(j++),
                c = bin.charCodeAt(j++);
            if ((a | b | c) > 255) throw new Error("String contains an invalid character");
            base64[base64.length] = table[a >> 2] + table[((a << 4) & 63) | (b >> 4)] +
                (isNaN(b) ? "=" : table[((b << 2) & 63) | (c >> 6)]) +
                (isNaN(b + c) ? "=" : table[c & 63]);
        }
        return base64.join("");
    },

    type: 'GitUtil'
};

Summary

We had to overcome a few hurdles to get our first KMF module working, but it was totally worth it! KMF and its various capabilities are an excellent improvement to instance security, and we look forward to exploring and leveraging the framework in the future.

Thanks for reaching here, hope this blog helps :)

Comments